interface-datastore

What is interface-datastore?

The `interface-datastore` npm package provides a set of interfaces and utilities for creating and working with datastores. It is designed to be used as a base for implementing various types of datastores, such as in-memory, file-based, or networked datastores. The package is part of the IPFS (InterPlanetary File System) project and is used to standardize how data is stored and retrieved.

What are interface-datastore's main functionalities?

Basic Operations

This feature allows you to perform basic CRUD (Create, Read, Update, Delete) operations on a datastore. The example demonstrates how to put, get, and delete a value in an in-memory datastore.

const { MemoryDatastore } = require('interface-datastore');
const datastore = new MemoryDatastore();

// Put a value
await datastore.put(new Key('/example'), Buffer.from('Hello, world!'));

// Get a value
const value = await datastore.get(new Key('/example'));
console.log(value.toString()); // 'Hello, world!'

// Delete a value
await datastore.delete(new Key('/example'));

Querying

This feature allows you to query the datastore for entries that match certain criteria. The example demonstrates how to put multiple values and then query the datastore for entries with a specific prefix.

const { MemoryDatastore } = require('interface-datastore');
const datastore = new MemoryDatastore();

// Put some values
await datastore.put(new Key('/example/1'), Buffer.from('Value 1'));
await datastore.put(new Key('/example/2'), Buffer.from('Value 2'));
await datastore.put(new Key('/example/3'), Buffer.from('Value 3'));

// Query the datastore
const query = datastore.query({ prefix: '/example' });
for await (const { key, value } of query) {
  console.log(key.toString(), value.toString());
}

Batch Operations

This feature allows you to perform multiple operations in a single batch, which can be more efficient than performing them individually. The example demonstrates how to create a batch, add operations to it, and then commit the batch.

const { MemoryDatastore } = require('interface-datastore');
const datastore = new MemoryDatastore();

// Create a batch
const batch = datastore.batch();

// Add operations to the batch
batch.put(new Key('/example/1'), Buffer.from('Value 1'));
batch.put(new Key('/example/2'), Buffer.from('Value 2'));
batch.delete(new Key('/example/3'));

// Commit the batch
await batch.commit();

Other packages similar to interface-datastore

level

The `level` package provides a simple, efficient, and flexible interface for working with LevelDB, a fast key-value storage library. It offers similar basic operations and querying capabilities but is specifically designed for use with LevelDB.

nedb

The `nedb` package is a lightweight, in-memory database with a MongoDB-like API. It offers similar CRUD operations and querying capabilities but is designed to be used as an embedded database for small applications.

pouchdb

The `pouchdb` package is a JavaScript database that syncs with CouchDB. It offers similar CRUD operations and querying capabilities but is designed for offline-first applications and synchronization with CouchDB.

README

interface-datastore

Implementation of the datastore interface in JavaScript

Lead Maintainer

Alex Potsides

Table of Contents

• Implementations
• Install
  • Test suite
  • Aborting requests
  • Concurrency
  • Keys
• API
• Contribute
• License

Implementations

• Backed Implementations
  • Memory: datastore-core/memory
  • level: datastore-level (supports any levelup compatible backend)
  • File System: datstore-fs
  • S3: datstore-s3
• Wrapper Implementations
  • Mount: datastore-core/src/mount
  • Keytransform: datstore-core/src/keytransform
  • Sharding: datastore-core/src/sharding
  • Tiered: datstore-core/src/tiered
  • Namespace: datastore-core/src/namespace

If you want the same functionality as go-ds-flatfs, use sharding with fs.

import FsStore from 'datastore-fs'
import { ShardingDataStore, shard } from 'datastore-core'

const fs = new FsStore('path/to/store')

// flatfs now works like go-flatfs
const flatfs = await ShardingStore.createOrOpen(fs, new shard.NextToLast(2))

Install

$ npm install interface-datastore

Test suite

Available via the interface-datastore-tests module

import { interfaceDatastoreTests } from 'interface-datastore-tests'

describe('mystore', () => {
  interfaceDatastoreTests({
    async setup () {
      return instanceOfMyStore
    },
    async teardown () {
      // cleanup resources
    }
  })
})

Aborting requests

Most API methods accept an AbortSignal as part of an options object. Implementations may listen for an abort event emitted by this object, or test the signal.aborted property. When received implementations should tear down any long-lived requests or resources created.

Concurrency

The streaming (put|get|delete)Many methods are intended to be used with modules such as it-parallel-batch to allow calling code to control levels of parallelisation. The batching method ensures results are returned in the correct order, but interface implementations should be thread safe.

import batch from 'it-parallel-batch'
const source = [{
  key: ..,
  value: ..
}]

// put values into the datastore concurrently, max 10 at a time
for await (const { key, data } of batch(store.putMany(source), 10)) {
  console.info(`Put ${key}`)
}

Keys

To allow a better abstraction on how to address values, there is a Key class which is used as identifier. It's easy to create a key from a Uint8Array or a string.

const a = new Key('a')
const b = new Key(new Uint8Array([0, 1, 2, 3]))

The key scheme is inspired by file systems and Google App Engine key model. Keys are meant to be unique across a system. They are typically hierarchical, incorporating more and more specific namespaces. Thus keys can be deemed 'children' or 'ancestors' of other keys:

• new Key('/Comedy')
• new Key('/Comedy/MontyPython')

Also, every namespace can be parameterized to embed relevant object information. For example, the Key name (most specific namespace) could include the object type:

• new Key('/Comedy/MontyPython/Actor:JohnCleese')
• new Key('/Comedy/MontyPython/Sketch:CheeseShop')
• new Key('/Comedy/MontyPython/Sketch:CheeseShop/Character:Mousebender')

API

https://ipfs.github.io/interface-datastore/

Contribute

PRs accepted.

Small note: If editing the Readme, please conform to the standard-readme specification.

License

MIT 2017 © IPFS